Repository navigation
Conversation
… statted A Bun.file() read that knows nothing about the file reads until read() returns 0. That last call is an EOF probe: it returns no bytes. A read after `.size` or `exists()` skips the probe, because the blob carries the cached stat size as its limit. A store can be statted while the size of the blob stays unknown: `.lastModified`, `Response.clone()` and `.body` on another wrapper all do that. Those reads paid the probe although the size of the file was known. ReadFile, ReadFileUV and FileReader now bound such a read by the size from the fstat that they already run at open. The bound applies only when the store was statted before the read, and the fstat reports a regular file with a size above 0. A size of 0 stays unbounded, because a procfs file reports 0 and has content. Read syscalls per operation on a 64 KiB file (/proc/self/io, debug build): .lastModified then text(): 2.04 -> 1.04 .lastModified then stream(): 2.10 -> 1.09 fresh text() and stream(): 2.03 and 2.10, unchanged .size then text() and stream(): 1.03 and 1.09, unchanged
No fstat runs when FileReader opens a file on Windows, so the stream reader had no size of its own there and could only stop at EOF. For a store that was already statted as a regular file, ask the open handle for its size (GetFileSizeEx: one call, no threadpool hop). The reader then stops at that size, as it does on POSIX.
|
Status: draft. This is the second PR of a stack: #43897 (refactor, green), this PR, then #43910 (the fix). What it changes, measured on Linux x64 with const f = Bun.file(p);
f.lastModified; // stats the store, the blob keeps no size
await f.text(); // main: 2.04 reads. here: 1.04Fresh reads (2.03) and reads after The new test in It stays a draft until the review of the whole stack is done. |
|
Closing: this change is now part of #43910. The reader bound moves the byte budget of a read from the cached #43910 has the same reader code with one difference: |
Problem
Bun.file()read that knows the file size skips the lastread()(0 bytes at EOF). Today only a read after.sizeorexists()knows it..lastModifiedorResponse.clone()has a statted store and no size on the blob. It pays thatread(): 2.04 pertext()of a 64 KiB file, not 1.03.st_sizeoff the blob (a procfs file reports 0 and has content). Every stat-first read would then pay it.Fix
ReadFile,ReadFileUVandFileReaderstop at the size from thefstatthey run at open. On WindowsFileReaderasks the handle (GetFileSizeEx).test/js/bun/util/bun-file-read.test.tscountsreadsyscalls (/proc/self/io): 2 on main, 1 here. More in Notes.Background
Blobover oneBun.file()shares a store that caches onestat. Onlyresolve_file_statwritesseekable, soseekable.is_some()means "statted".exists(), a write, thentext()stops at the old length (BunFile.text()does not return correct content after BunFile.write()#23902). A freshfstatcosts the same.Downsides
fstatand the lastread()yields itsfstat-time bytes for these reads. A read after.sizealready stops at an older size.GetFileSizeExper stream of a statted store. POSIX: no new syscall.Notes
readsyscalls per operation, 64 KiB file,/proc/self/io(syscr) over 300 iterations, debug builds, Linux x64. Syscall counts do not depend on the build profile.text().size, thentext()exists(), thentext().lastModified, thentext()stream().size, thenstream().lastModified, thenstream()new Response(file).bodyWindows x64, read operations per operation from
Win32_Process.ReadOperationCount, same file, 300 iterations, debug builds:.lastModifiedthentext()2.01 to 1.01,.lastModifiedthenstream()2.01 to 1.01. Fresh reads stay at 2.01 and stat-first reads stay at 1.01.The gate is
file_store.seekable.is_some()on theFilethat the reader captured when it was created. A freshBun.file(p).text()keeps its behaviour: it reads until EOF, and it returns bytes that were appended after itsfstat.A file whose
fstatunder-reports a size above 0 is cut at that size for these reads, as.sizethentext()already is on main. A size of 0 (procfs, sysfs with no content length) is never a bound.The plan for this stack called this PR a refactor with no behaviour change. That is not exact, because of the first Downsides line, so this PR has a test and does not claim it.
Suites run with a debug build, Linux:
blob,body,body-clone,wasm-streaming,structured-clone-blob-file(1044 pass),streams,fetch-file-upload,bun-file-fd-read,empty-file(638 pass).fetch.test.ts,bun-file-exists,bun-file,bun-file-read,bun-serve-file,FormData: the same 21 failures by name as main (they need IPv6, the public internet, or a non-root user). Windows: 1719 pass in 13 suites.bun run rust:check-allpasses on 12 targets.